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1.0 


INTRODUCTION 


The  motivation  for  this  proposed  research  is  that  the  volume,  variety,  and  complexity  of  data 
being  collected  by  the  intelligence,  reconnaissance,  and  surveillance  community  have  far 
exceeded  traditional  signal  extraction  procedures  for  anomaly  detection.  Knowing  the  proper 
signature  (defined  as  an  ensemble  of  features  that  correspond  to  a  given  target  or  activity)  to 
extract  from  a  signal  or  image  is  the  key  to  all  intelligence  data  processing.  However,  most 
signatures  found  relevant  for  one  scale  or  resolution  might  lose  their  meaning  when  the 
resolution  is  changed,  which  is  typically  the  case  when  the  sensing  field-of-view  or  area  of 
coverage  changes.  For  example,  a  visually  familiar  posture  or  gesture  will  change  or  even 
become  invisible  when  viewed  from  far  away.  Under  the  Air  Force  Research  Laboratory’s 
(AFRL)  Layered-Sensing  construct,  target  size  can  vary  greatly  depending  on  the  particular 
sensor  that  is  collecting  data.  It  is  very  possible  that  a  signature  that  is  useful  in  extracting 
target  or  activity  in  one  sensor  resolution  may  become  useless  in  another.  A  size-invariant 
signature  would  make  data  collection  and  processing  more  efficient. 
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2.0 


OBJECTIVE 


For  the  first  time,  we  can  see  all  the  rhythms  and  melodies  of  day-to-day  human  activities  under 
one  view  for  very  long  duration.  This  capability  of  a  sensor  to  loiter  persistently  over  the  same 
area  creates  an  opportunity  to  observe  and  find  possible  anomalous  activities  that  need  a  further 
look  by  a  higher  resolution  sensor,  which  might  possibly  result  in  a  corresponding  change  of  the 
observable  signature.  In  these  wide-area  staring  sensors,  a  person  would  subtend  at  most  a  few 
pixels  on  the  image  plane  without  showing  any  spatial  features.  However,  everyone’s 
movements  could  still  be  registered  as  trajectories.  Assuming  a  trajectory  is  the  physical 
manifestation  of  one’s  underlying  intent  (either  malicious  or  non-malicious),  it  could  be  used  as 
the  basis  for  a  suitable  “behavior  signature”  for  the  underlying  intent.  (A  behavior  signature  is 
defined  as  an  activity  or  activities  that  can  be  geo-spatially  connected  to  an  event  and  which,  in 
turn,  can  be  used  to  mathematically  determine  the  source  event.)  Thus,  a  person’s  motion 
trajectory,  no  matter  how  registered  by  a  sensor  or  how  many  pixels  are  involved,  might  directly 
reflect  that  person’s  malicious  intent  or  anomalous  activity,  and  thereby  allow  harvesting  more 
than  just  pictorial  information  from  wide-area  persistent  sensors.  In  general,  the  path  or  position 
of  any  object  in  time  is  a  result  of  the  underlying  process  or  processes  that  generate  the  motion. 

It  is  hypothesized  that,  by  knowing  the  2D  path  and  temporal  trajectory  of  an  object  and 
converting  it  to  a  ID  time  series,  the  process(es)  responsible  for  different  human  activities  may 
be  deduced  from  the  quantitative  representation  of  the  trajectory  in  both  the  time  and  frequency 
domains  as  a  mathematical  signature  characterized  by  a  series  of  transfer  function  equations  and 
associated  scaling  exponents. 
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3.0 


APPROACH/ANALYSIS 


As  a  proof  of  concept  of  the  ability  of  human-motion  trajectory-only  data  to  yield  information 
about  the  nature  of  the  underlying  activity  without  prior  knowledge  about  who  or  how  someone  is 
moving,  time-series  data  were  collected  for  four  different  walking  scenarios:  nonnal  walking, 
walking  around  picking  up  sticks,  walking  while  stalking  another  person,  and  walking  openly 
while  surveilling  an  area  but  avoiding  being  conspicuous. 

3.1  Data  Collection 

Specifically,  in  order  to  better  understand  the  classification  of  different  kinds  of  movement 
using  the  scaling  exponent,  four  scripted  walking  actions  were  recorded  using  a  Global 
Positioning  System  (GPS)-enabled  BlackBerry  (BB)  smartphone.  GPS  data,  consisting  of 
latitude  and  longitude  infonnation,  was  recorded  at  approximately  two-second  intervals 
capturing  the  total  path  of  a  subject,  and  was  converted  to  a  time  series  as  the  change  in  step 
size  (distance)  per  unit  time.  Two  open-air  locations  were  selected  with  two  scripted  walking 
tasks  occurring  at  each  location.  At  each  location,  the  second  walking  task  immediately 
followed  the  first  walking  task,  but  both  events  at  a  location  were  recorded  as  one  data  set. 
However,  each  walking  task  was  entirely  independent.  That  is,  there  was  no  overlap,  and  the 
transition  from  one  walking  task  to  the  next  was  abrupt.  Further,  at  no  time  after  transitioning 
to  the  second  scripted  walking  task  did  the  script  revert  to  the  first  task. 

Data  Analysis.  Once  the  data  were  recorded,  the  latitudes  and  longitudes  were  converted  to  a 
single  time  series.  Although  the  2D  bird’s-eye  view  of  the  path  and  trajectory  time-series 
appear  uniform  (especially  when  the  points  are  connected),  the  conversion  to  a  ID  time  series 
(dropping  location  data  and  substituting  distance-travelled  per  time  interval)  offers  both  visual 
and  mathematical  cues  of  two  distinct  behaviors.  For  each  location,  the  first  part  of  the  time 
series  representing  one  scripted  action  appears  visually  distinguishable  from  the  second  part 
representing  a  new  motion  type  (see  Figures  2  and  3).  The  visual  difference  can  be  expressed 
mathematically.  In  turn,  the  mathematical  change  in  time-series  behavior  indicates  a  change  in 
the  physical  process  or  purposeful  intent  behind  the  generation  of  the  time  series.  This  physical 
change  is  reflected  in  a  change  in  the  observed  scaling  behavior  in  the  data.  If  the  data  of  two 
behaviors  are  combined,  the  more  dominant  behavior  will  have  a  greater  influence  on  the 
measured  scaling  exponent.  Therefore,  a  sliding  window  approach  is  preferred  in  which  a  time 
series  is  passed  through  a  preset  window  which  is  constantly  evaluating  the  time  series  over 
the  length  of  the  window  and  looks  for  a  change  in  the  scaling  exponent  as  an  indication  of  a 
change  to  a  new  behavior. 
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4.0  RESULTS 

The  results  for  the  two  locations  each  containing  two  scripted  walking  tasks  are  as  follows: 

Location  1  Activity:  Two  Very  Different  Sequential  Walking  Tasks: 

Walking  Around  a  Parking  Lot  /  Picking  up  Sticks  (Figure  1):  Case  lconsists  of  two  grossly 
different  walking  patterns,  namely,  walking  around  a  parking  lot  versus  picking  up  sticks.  The 
first  338  points  representing  the  walking  phase  (left  side  in  Figure  2)  exhibit  a  slightly  different 
scaling  behavior  and  scaling  exponent  than  the  action  of  picking  up  sticks  (right  side  in  Figure 
2).  Using  a  sliding  window,  the  time  series  of  distance  travelled  per  unit  time  changes 
suddenly  at  Sample  Number  338.  Note  how  the  minimum  distance  travelled  in  each  step  for 
the  first  338  points  is  above  one  meter,  on  average,  per  sampling  interval  suggesting  continuous 
movement  (walking).  After  point  339,  the  minimum  and  maximum  distance  fluctuates  more 
wildly  indicating  more  variable  movement  (picking  up  sticks).  At  some  times,  there  is  no 
movement  as  the  individual  stopped,  and  the  distance  travelled  is  0  m  for  a  sampling  interval. 


Figure  1:  Overhead  View  of  Case  1  Scenario 

Blue  dots  are  normal  walking  path  locations  and  red  dots  are  locations  of  picking  up  sticks  activity 
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dettaDistance  of  Path  from  X  and  Y 
after  the  first  two  samples  are  omitted 


Sample  Number  (-2  second  intervals)  Sample  Number  (-  2  second  intervals) 

Add  +339  to  this  Sample  Number  to  obtain  relationship  to  first  338  points 

Figure  2:  Time  Series  of  Distance  Travelled  at  each  Two-Second  Interval  for  the  Normal 

Walking  and  Picking  Up  Stick  Phrases  of  Case 

The  mechanism  that  is  causing  the  observed  change  in  scaling  behavior  needs  to  be  addressed.  In 
walking,  movement  does  not  come  to  a  complete  stop  and  is  the  continuous  integration  of  more 
or  less  a  relatively  constant  velocity.  As  a  consequence,  the  step  size  rarely  falls  below  1  m  per  2 
second  interval.  Continuous  movement  may  have  noise  at  only  the  high  frequencies:  movement 
at  an  average  or  fixed  speed  is  considered  “normalized”  around  that  speed  so  that  a  constant 
velocity  is  stable  as  fluctuations  around  a  constant  velocity  consist  of  low  amplitude,  high 
frequency  noise.  The  result  is  that  the  scaling  exponent  is  low  and  close  to  that  which  is 
considered  to  be  the  completely  random  behavior  of  Gaussian  white  noise.  However,  this  is  not 
to  say  that  action  of  walking  or  the  purposeful  intent  (input  velocity)  of  walking  is  random. 

Instead,  the  interpretation  is  that  any  constant  velocity  spectrum  is  nearly  the  same  as  that  for 
Gaussian  white  noise  with  small  high  frequency  fluctuations.  The  motion  is  otherwise 
unchanging  with  frequency  and  therefore  generates  a  remarkably  flat  spectrum  similar  to  that  of 
random  behavior. 

The  change  in  the  scaling  exponent  of  the  second  scripted  motion  (right  side  in  Figure  2)  in  the 
latter  half  of  the  Location  1  activity  indicates  a  change  in  the  behavior  of  the  individual 
performing  the  movement.  In  an  activity  such  as  picking  up  sticks  on  a  property,  there  is 
increased  variability  based  on  the  location  of  the  stick  and  the  pile  to  which  the  stick  is  moved 
combined  with  the  stuttered  back  and  forth  movement  of  the  work.  The  increased  variability,  at 
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this  resolution,  is  through  reduced  randomness,  since  there  is  more  purpose  and  hence 
coordinated  movement  behind  the  task  of  picking  up  sticks.  From  the  change  in  step  size  per 
unit  time,  a  change  point  was  able  to  be  determined  corresponding  to  an  abrupt  change  in  scaling 
behavior  at  Point  338  indicating  a  different  physical  process  taking  place  in  the  time  series. 

From  a  filtering  perspective,  from  the  change  in  the  scaling  exponent  alone,  the  two  scripted 
actions  within  the  data  can  be  accurately  separated  with  the  first  338  points  defining  a  continuous 
walk  around  the  parking  lot  and  the  second  portion  from  Point  339  to  the  end  of  the  time  series 
defined  as  picking  up  sticks  on  a  property.  Additionally,  the  duration  of  each  activity  can  easily 
be  calculated.  For  example,  the  first  338  points  at  approximately  2-  second  intervals  translates  to 
a  total  of  676  seconds  or  a  walk  around  the  parking  lot  that  took  roughly  1 1  minutes  and  16 
seconds  (if  the  sampling  interval  was  constant).  Potentially,  in  knowing  the  scaling  exponent, 
the  transfer  function,  and  the  duration  of  an  activity,  a  catalog  of  actions  may  be  created  as  a 
filter  bank  for  unknown  data. 

Location  2  Activity:  Two  Similar  Sequential  Walking  Tasks: 

Stalking  Target  /  Active  Walking  Surveillance  (Figure  3):  Location  2  activity  consists  of  two 
subtly  similar  scripted  actions  representing  stalking  behavior  (left  side  in  Figure  4)  and  active 
walking  surveillance  (right  side  in  Figure  4)  in  and  around  the  parking  lot  to  assess  the  area. 
Although  the  distinction  between  the  scaling  exponents  is  small,  the  scaling  exponent  does 
distinguish  between  the  two  distinct  scripted  actions  which  are  very  similar  behaviors 
involving  movement.  The  difference  between  the  two  behaviors  is  how  one  moves  when 
stalking  someone  so  as  not  to  be  seen  by  the  target  versus  how  one  moves  when  assessing  an 
area  and  performing  surveillance  after  the  target  has  left  the  area. 


Figure  3:  Overhead  View  of  Case  2  Scenario 

Blue  dots  are  stalking  path  locations  and  red  dots  are  locations  surveillance  walking  while  trying  to  be 

inconspicuous 
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DeltaDistance  (Change  in  Step  Size  with  each  Sampling  Interval)  of  Path 


Sampling  Interval  Sampling  Interval 


Figure  4:  Time  Series  of  Distance  Travelled  at  each  Two-Second  Interval  for  the  Stalking 

and  Surveillance  Walking  Phases  of  Case  2 

From  the  time  series  of  the  stalking  action,  one  observes  that  the  change  in  distance  (velocity) 
with  each  time  increment  is  very  stuttered  with  sudden  increases  in  velocity  to  catch  up  with  a 
target  followed  by  moderately  long  periods  of  inactivity  (approximately  10  to  20  seconds  on 
average)  upon  reaching  cover.  The  bursts  of  velocity  result  in  a  change  in  location  as  much  as 
6  m  per  event  and  because  more  time  is  spent  in  high  frequency  but  low  amplitude  situations 
(under  cover)  interspersed  with  sudden  large  amplitude  but  lower  frequency  runs. 

Comparatively,  in  active  surveillance  when  a  target  is  no  longer  present,  the  increased 
freedom  of  movement  moderates  the  time  series  allowing  more  or  less  an  average  velocity  of 
around  just  under  2  m  per  unit  time  (or  1  m/s  for  2  second  intervals).  This  lowers  the  scaling 
behavior  as  velocity  experiences  normalization.  However,  the  scaling  exponent  is  higher  than 
that  for  walking  in  Case  1  since  there  are  still  staggered  starts  and  stops  to  both  acquire 
infonnation  and  assess  the  area  without  looking  conspicuous. 
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5.0 


DISCUSSION  OF  PRELIMINARY  RESULTS 


As  promising  as  the  results  of  this  proof-of-concept  demonstration  are  for  unlocking  the 
information  contained  in  purely  trajectory-only  data,  there  are  however,  some  limitations  to 
using  the  scaling  exponent  and  transfer  functions  to  define  human  movement. 

Specifically,  one  limitation  is  the  degree  to  which  one  is  able  to  deduce  intent  from  pure 
trajectory  data.  In  order  to  do  so,  one  would  need  to  assume  that  for  every  given  type  of 
motion,  there  could  be  identified  a  unique  and  corresponding  state  of  intention.  However,  there 
are  likely  to  exist  very  similar  movement  patterns  for  which  the  underlying  intentions  would 
be  very  different,  such  as  picking  up  sticks  that  were  scattered  about  versus  searching  for  a  lost 
cat.  There  are  also  likely  to  be  very  similar  states  of  intention  for  which  the  corresponding 
movement  patterns  would  be  very  different,  such  as  avoiding  immanent  detection  by  quickly 
walking  away  versus  hiding  in  a  building. 

Nonetheless,  this  approach  to  inferring  intent  from  movement  patterns  does  show  promise  and 
it  remains  to  be  determined  whether  the  capability  to  infer  intent  from  pure  trajectory-only  data 
might  be  improved  by  considering  the  context  within  which  the  movement  occurs. 

More  lines  of  evidence  may  be  needed  to  accurately  describe  a  specific  behavior  such  as  the 
duration  of  an  event,  the  range  of  the  time  series,  and  even  the  location  of  an  event  which 
defined  what  behaviors  are  likely  or  even  possible  in  an  area.  For  example,  dancing  and 
skipping  are  unlikely  along  a  steep  rocky  slope.  Together,  time  series  movement  may  be 
summarized  through  a  combination  of  some  or  all  of  these  parameters  to  ensure  the  best 
possible  mathematical  description  of  a  behavior  or  the  purposeful  intent  behind  a  behavior. 

Collection  and  analysis  of  GPS  data  for  walking  three  designated  routes  on  the  campus  of 
Wright  State  University  by  volunteer  students  are  continuing. 
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LIST  OF  ACRONYMS 


AFRL  Air  Force  Research  Laboratory 

BB  BlackBerry 

GPS  Global  Positioning  System 
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